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ABSTRACT 

Historieally,  Army  aequisition  has  had  diffieulty 
eondueting  an  adequate  early  assessment  of  the  human 
dimension  in  system  performanee.  Proaetive  researeh  on 
human  performanee,  however,  is  vital  to  aehieving  the 
unpreeedented  allianee  of  humans  and  maehines 
antieipated  with  Future  Combat  Systems  (FCS).  This 
paper  summarizes  researeh  methods  and  findings  aeross 
four  exploratory  experiments  foeused  on  the  eommand 
group  of  a  small  eombined  arms  unit  eomposed  primarily 
of  unmanned  air  and  ground  vehieles.  Results  are  based 
on  highly  detailed  objeetive  measures  of  verbal  and 
human-eomputer  interaetion  and  an  array  of  subjeetive 
measures  from  expert  and  noviee  partieipants.  Findings 
underseore  potential  problems  in  training  and  workload 
with  FCS,  and  potential  solutions  through  user-based 
involvement  and  proaetive  researeh  to  ensure  teehnology 
eomplements  human  performanee. 

1.  INTRODUCTION 

The  U.  S.  Army’s  transformation  to  Future  Combat 
Systems  (FCS)  antieipates  an  extraordinary  allianee  of 
humans  and  maehines.  Examples  inelude  humans 
working  with  “hots”  to  proeess  information  and  make 
deeisions,  and  working  with  robots  to  move,  see,  and 
strike  (Department  of  the  Army,  2003).  Creating  an  FCS 
allianee  that  aetually  improves,  and  does  not  impede, 
battle  eommand  is  a  human  systems  integration  ehallenge. 

A  fundamental  lesson  from  modem  warfare  is  that 
the  insertion  of  teehnology  burdens  and  stresses  the  foree 
(Cordesman  &  Wagner,  1996).  The  burden  on  Soldiers 
with  new  teehnology  is  attributed  less  to  teehnology  per 
se,  than  to  inflated  expeetations  about  teehnology. 

New  teehnologies  may  help  eommanders  visualize 
and  deseribe  the  operation  and  direet  subordinates  to 
mission  aeeomplishment.  However,  many  eommand  and 
eontrol  (C^)  tasks  are  too  eomplex  and  important  to  assign 
to  maehines.  How  to  best  alloeate  human-maehine  tasks 
and  funetions  for  future  eommand  groups  is  a  key  eoneem 
in  Army  transformation. 


The  Army  learns  by  doing.  The  researeh  reported 
here  exemplifies  the  Army’s  effort  to  proaetive ly  explore 
C^  eoneepts  and  address  human  system  integration  issues 
for  the  Future  Foree.  The  FCS  C^  researeh  program, 
eurrently  ealled  Multi-Cell  and  Dismounted  Command 
and  Control  (MC&D  C^),  eondueted  an  ineremental  series 
of  eommand-in-the-loop  experiments  at  Fort  Monmouth, 
NJ  from  Oetober  2001  to  Mareh  2003. 

Program  leads  were  the  Defense  Advaneed  Researeh 
Projeets  Ageney  (DARPA)  and  the  U.S.  Army 
Communieations-Eleetronies  Command  (CECOM) 
Researeh,  Development  and  Engineering  Center  (RDEC). 
As  a  program  partner,  the  U.S.  Army  Researeh  Institute 
for  the  Behavioral  and  Soeial  Seienees  (ARI)  measured 
human  performanee  to  improve  human  system  integration 
and  training,  and  to  support  the  Seienee  and  Teehnology 
Objeetive  (STO)  titled  “Methods  and  Measures  of 
Commander-Centrie  Training.” 

The  purpose  of  the  FCS  C^  researeh  was  to  explore 
the  hypothesis  that  digitization  of  eurrent  battlefield 
operating  systems  enables  a  new  approaeh  to  eommand 
and  eontrol.  To  test  this  hypothesis,  the  FCS  C^  program 
ereated  a  researeh  environment  to  assess  eommand  group 
performanee  in  a  notional  FCS  small  eombined  arms  unit, 
ealled  the  Unit  Cell.  Figure  1  depiets  the  manned  and 
robotie  elements  of  the  Unit  Cell  organization  used  in 
Experiments  1-4,  ineluding  the  surrogate  C^  vehiele 
oeeupied  by  the  eommand  group  partieipants.  The 
eommand  group  direetly  eontrolled  16  air  and  ground 
vehiele  systems,  13  of  whieh  were  unmanned  platforms. 

The  four  primary  partieipants  in  the  eommand  group 
were  aetive-duty  U.  S.  Army  lieutenant  eolonels  who 
served  as  Commander,  Information  Manager,  Effeets 
Manager,  and  Battle  Spaee  Manager.  Their  four  C^ 
workstations  provided  a  eommon  pieture  of  the  battlefield 
and  allowed  them  to  eommand  and  eontrol  their  Unit  Cell 
assets.  Through  the  C^  workstations  linked  to  Distributed 
Interaetive  Simulation  (DIS),  the  eommand  group 
partieipants  interaeted  with  virtually  simulated  elements 
of  the  friendly  foree,  the  threat  foree,  and  eivilian/ 
noneombatant  entities  during  the  eonduet  of  experimental 
missions. 
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Figure  1 .  Organization  of  the  Unit  Cell 

To  explore  new  approaches  to  command  and  control, 
the  research  environment  and  the  command  group’s 
operational  requirements  were  expanded  and  refined  as 
the  experiments  progressed.  New  features  were  added  to 
the  prototype  and  older  features  were  refined  or 
abandoned,  based  on  lessons  learned  in  prior  experiments. 
These  technical  refinements  often  resulted  in  changes  by 
the  command  group  in  the  allocation  of  tasks  and 
functions. 

By  design,  the  operational  requirements  on  the 
command  group  and  the  Unit  Cell  escalated  across 
experiments,  as  bulleted  below.  Experiment  1  only 
addressed  the  ability  of  the  Unit  Cell  to  See/Move, 
namely  to  maneuver  its  elements  in  order  to  see  and  not 
be  seen.  By  Experiment  4,  the  mission  of  the  Unit  Cell 
and  capabilities  of  the  C^  prototype  progressed  to 
Improved  See/Move/Strike  and  Sustain  operations. 

■  Exp  1  —  Dec  01  —  See/Move  (with  limited  Strike). 

■  Exp  2  —  May  02  —  Improved  See/Move  and  Strike. 

■  Exp  3  —  Sep  02  —  Improved  See/Move/Strike. 

■  Exp  4  —  Mar  03  —  Improved  See/Move/Strike  and  Sustain. 

An  additional  experiment  with  Army  Cadets  called 
the  Summer  Experiment  was  conducted  in  August  2002 
that  resulted  in  a  comparison  of  novice  versus  expert 
command  group  performance. 

In  support  of  the  FCS  C^  effort,  ARI  developed  and 
applied  a  set  of  human  performance  measurement 
methods  to  systematically  describe  and  quantify  the 
behavior  of  the  command  group.  This  document  provides 
a  brief  overview  of  the  methods,  results,  and  conclusions 
of  the  ARI  effort. 

More  complete  documentation  is  in  ARI’s  Interim 
Reports  on  each  experiment  available  from  the  Program 
Manager  (PM)  MC&D  C^  and  ARI  reports  (Lickteig, 
Sanders,  Durlach,  Fussier,  &  Carnahan,  2004a;  Lickteig, 
Sanders,  Durlach,  &  Carnahan,  2004b). 


2.  METHOD 

For  each  two-week  experiment,  the  command  group 
typically  conducted  10-12  Deliberate  Attack  missions 
called  experimental  “runs”  in  a  mock-up  C^  vehicle, 
shown  in  Figure  2.  Each  experimental  run  required 
approximately  2-3  hours  for  completion  including  the 
planning  and  execution  phase,  resulting  in  two  runs  per 
day.  At  the  end  of  each  day,  a  team  of  subject  matter 
experts  led  an  After  Action  Review  (AAR)  of  the  day’s 
runs  to  review  operational,  technical,  and  human 
performance  issues. 


Figure  2.  Command  group  participants  in  a  surrogate 
command  and  control  (C^)  vehicle. 

Table  1  depicts  the  basic  experimental  schedule  for 
the  FCS  C^  Experiments  1-4.  Variations  in  schedule  were 
relatively  moderate  for  exploratory  research  and  mainly 
due  to  technical  problems  during  Experiments  1  and  2. 
The  first  three  days  of  each  experiment  were  dedicated  to 
training,  and  the  seven  remaining  days  to  experimental 
runs.  Training  on  days  1-2  addressed  participants’ 
individual  skills  with  the  C^  prototype  system.  Training 
was  intentionally  not  duty  position  specific,  but  rather 
designed  to  provide  cross-duty  skills  across  the  command 
group.  Collective  training  on  day  3  took  place  in  the  C^ 
vehicle  and  used  virtual  simulation  to  support  run 
rehearsal. 


Table  1.  Basic  Schedule  For  Experiments  1-4. 
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2.1  Experimental  Design 

A  key  feature  of  the  experimental  design  was  a 
deliberate  practice  approach  in  which  a  stable  group  of 
experienced  officers  repeatedly  executed  a  similar 
mission  on  similar  terrain  with  similar  assets.  The  use  of 
deliberate  practice  helped  insure  results  were  based  on  a 
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proficient  command  group.  A  common  mistake, 
especially  when  experimental  systems  differ  radically 
from  current  systems,  is  testing  with  unfamiliar  troops 
who  receive  last-minute  training.  In  such  cases,  training 
deficiencies  may  overshadow  and  confound  all  other 
performance  considerations. 

A  second  design  feature  involved  systematically 
varying  run  complexity  levels  as  a  function  of  METT-TC 
(Mission,  Enemy,  Terrain,  Troops,  Time  and  Civilians). 
Three  levels  of  run  complexity  (Medium,  High,  and  Too 
High)  were  developed  by  increasing  enemy  force  activity 
and  size,  restricting  a  key  friendly  asset,  and  inserting 
civilians  on  the  battlefield.  Levels  of  complexity  were 
randomly  assigned  to  runs  during  Experiments  1-3, 
however,  all  runs  were  conducted  at  the  High  level  during 
Experiment  4  to  establish  “baseline”  performance 
indicators  for  future  efforts. 

Run  complexity  was  used  to  identify  how  changes  in 
the  unit’s  operational  conditions  affected  command  group 
perfomiance,  and  to  gauge  the  performance  limits  of  the 
proposed  Unit  Cell  organization.  The  complexity  of  the 
mission  was  expected  to  change  workload,  performance, 
and  human-system  integration  requirements. 

2.2  Interface  Design 

Figure  3  provides  a  sample  depiction  from  one  of  the 
two  C^  displays  at  each  participant  workstation.  The  dual 
displays  were  identical  and  allowed  participants  to 
independently  configure  each  display  to  support  C^ 
requirements  by  position  and  user  preferences.  Each 
display  provided  a  common  picture  of  the  battlefield 
situation  and  control  over  all  the  automated  and 
interdependent  systems  in  the  Unit  Cell. 


Figure  3.  A  sample  view  of  a  C^  display  with  the  sensor 
Image  Viewer  opened  at  the  left. 


2.3  Human  Performance  Measures 

Four  types  of  human  performance  measures  were 
developed,  iteratively  refined,  and  repeatedly  used  during 
Experiments  1-4: 


■  Verbal  interaction  measures  based  on  audio  recordings 
of  the  exercises  examined  the  command  group’s  verbal 
communications. 

■  Human-Computer  Interaction  (HCI)  measures  based  on 
video  recordings  of  the  exercises  examined  the 
command  group’s  interactions  with  their  prototype  C^ 
systems. 

■  Subjective  measures  included  surveys  and 
questionnaires  soliciting  ratings  and  comments  from  the 
command  group  participants. 

■  Automated  measures  were  designed,  and  a  few 
developed  and  validated,  to  automatically  record  the 
command  group’s  interactions  with  the  prototype  C^ 
systems  during  experimental  runs. 

2.4  Evaluation  Framework 

The  human  performance  research  methods  used  an 
evaluation  framework  based  on  a  functional  analysis  of 
the  participants’  command  group  performance.  The  set  of 
C^  functions  used  to  analyze  verbal  and  human-computer 
interactions  were: 

■  Plan:  Develop,  assess,  and  modify  plans  including 
tasking  for  unmanned  air/ground  assets  in  response  to 
changing  events. 

■  See:  ControFinterpret  input  from  a  set  of  manned/ 
unmanned  networked  sensors  to  maintain  accurate 
battlefield  “picture.” 

■  Move:  Control  movement  and  activity  of  manned/ 
unmanned  assets. 

■  Strike:  Control  manual/automated  networked  fires. 

■  BDA:  Control/interpret  input  from  a  set  of  manned/ 
unmanned  networked  sensors  for  information  collection 
and  battle  damage  assessment  (BDA). 

3.  RESULTS 

A  small  sample  of  the  results  on  human  performance 
across  Experiments  1-4  is  provided  in  this  section. 

3.1  Verbal  Communications  Results 

Verbal  interaction  by  the  command  group  was  an 
almost  continuous  activity.  Breaks  in  command  group 
verbalization,  or  silent  time,  averaged  only  7%  during  the 
execution  phase  of  a  run.  The  finding  that  verbalization 
by  the  command  group  occurred  93%  of  the  time  during 
mn  execution  is  noteworthy.  The  pattern  of  steady 
conversation  occurred  despite  participants’  common 
access  to  a  visually  rich  and  timely  depiction  of  the 
battlefield  situation  on  their  C^  displays.  Figure  4 
illustrates  the  dominant  role  of  the  Commander  in  the 
command  group’s  discussions  across  the  four 
experiments. 
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Figure  4.  Percent  of  verbalization  by  duty  position. 


The  largest  percentage,  about  30%,  of  verbalizations 
during  run  execution  supported  the  See  Function,  as 
shown  in  Figure  5.  Verbal  discussion  primarily  supported 
collection  and  interpretation  of  data  from  multiple  ground 
and  air  sensors  to  construct  an  accurate  picture  of  the 
battlefield.  Nearly  15%  of  all  discussion  was  devoted  to 
the  closely  related  function  of  battle  damage  assessment 
(BDA). 


Figure  5.  Percent  of  verbalization  by  function. 

In  addition  to  duty  position  and  function,  the 
communications  within  the  command  group  and  with 
surrogate  higher  echelons  were  analyzed  by: 

■  Source:  Within  Cell,  Cell-Higher,  Higher-Higher 

■  Factor:  Mission,  Enemy,  Tenain,  Troops,  Time,  &  Civilians 

■  Type:  Share,  Act,  Direct,  Ask,  Process,  and  Decide 

■  System:  Platform  break-out  of  FCS  air/ground  assets. 

3.2  Human-Computer  Interaction  (HCI)  Results 

The  coding  categories  developed  for  HCI  analysis 
were  based  on  the  C^  functions  of  Plan,  See,  Move  and 
Strike.  The  categories  and  codes  used  to  assess  HCI  were 
revised  and  expanded  over  the  course  of  experiments  as 
new  features  were  added  to  the  C^  prototype.  For 
example,  thirteen  (13)  new  features  were  introduced  for 
Experiment  3,  such  as  automated  fires  based  on  sensor- 
shooter  links  and  automated  route  generation  for 
unmanned  vehicles. 

HCI  analysis  also  identified  and  related  sub-functions 
and  interactions  to  the  primary  C^  functions,  as  depicted 
in  Figure  6.  For  Experiments  2-4  respectively,  the 
number  of  HCI  categories  expanded  from  53  to  84  to  97 


Conuniind  &  Control 


Figure  6.  HCI  coding  categories  by  Experiment  4. 


different  types  of  human  interaction  with  the  C^ 
prototype.  (No  HCI  analysis  of  Experiment  1  was 
possible  due  to  low-resolution  video  recordings.) 


Figure  7  provides  a  summary  look  at  HCI  frequency 
during  typical  90-minute  run  execution  phases  by  duty 
position  and  experiment.  The  frequency  of  interaction  by 
the  three  Battle  Managers  during  run  execution  was 
substantially  higher  than  the  Commander’s  interaction. 
Peak  performance  by  the  Battle  Managers  during  more 
intense  run  segments  was  9-12  interactions  per  minute. 


Duty  Position 


□  Experiment  2 

□  Experiment  3 
■  Experiment  4 


Figure  7.  Frequency  of  interaction  during  run  execution 
by  duty  position  and  experiment. 

Figure  8  provides  a  more  detailed  look  at  HCI  results 
by  C^  function,  duty  position,  and  display  during  the 
execution  phase  of  a  typical  run.  The  1,044  interactions 
summarized  in  Figure  8  related  to  the  following  C^ 
functions:  Move  (5%),  See  (75%),  and  Strike  (20%).  By 
comparison,  the  499  interactions  during  the  planning 
phase  of  the  same  run  were  related  to  the  following 
functions:  Plan  (36%),  Move  (11%),  See  (51%),  and 
Strike  (2%). 
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Figure  8  also  illustrates  how  each  participant  elected 
to  perform  their  interactions  by  display.  Display 
preferences  were  clearly  not  unifomi:  left  display  used 
primarily  by  the  Effects  Manager,  right  display  by  the 
Battlespace  Manager,  and  fairly  balanced  left  and  right 
display  use  by  the  Commander  and  the  Information 
Manager.  Such  results  raise  research  questions  about 
interface  design,  including  user  needs  and  preferences, 
that  future  efforts  might  address  to  improve  human 
system  integration. 


Function  by  Duty  Position 


Figure  8.  Frequency  of  interaction  by  function,  duty 
position,  and  display  during  a  run  execution  phase. 


3.3  Subjective  Results 

Subjective  results  addressed  many  research  issues 
including  workload,  human  perfomiance,  system 
perfomiance,  and  training.  Workload  was  a  key  concern 
given  the  FCS  goal  of  reducing  the  size  of  the  command 
group  for  a  small  unit  with  manned  and  predominantly 
robotic  elements. 


Figure  9  provides  a  comparison  of  each  participant’s 
average  rating  of  overall  workload  across  complexity 
levels  by  experiment  and  duty  position.  The  results 
indicate  that  participants  experienced  moderate  to  high 
levels  of  workload  with  the  highest  workload  reported  by 
the  Information  Manager. 
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Battlespace 

.  . 

Information 

Effects 
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Figure  9.  Overall  workload  ratings  by  duty  position  and 
experiment  (0  =  Very  Low,  100  =  Very  Fligh). 

Performance  success  is  always  a  key  concern. 
Participant  ratings  of  performance  success  during  run 
execution  declined  sharply  at  the  Too  Fligh  level  of 
complexity,  as  shown  in  Figure  10  from  Experiment  3. 


The  Information  Manager’s  low  ratings,  for  example, 
reflect  human  system  integration  issues  in  controlling 
unmanned  aerial  vehicles  (UAVs)  and  in  analyzing  more 
sensor  images  in  the  Too  Fligh  condition. 
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Figure  10.  Mean  performance  success  ratings  by  run 
complexity  and  duty  position  during  a  typical  run’s 
execution  phase  (0  =  Failure,  100  =  Perfect). 


System  performance  ratings  by  participants  assessed 
the  effectiveness  of  key  prototype  features  added  or 
modified  across  experiments.  Figure  11  indicates  how 
participants  rated  the  thirteen  (13)  new  features  added  for 
Experiment  3.  Overall,  most  of  the  new  features  received 
positive  ratings.  Flowever,  ratings  were  “Ineffective”  to 
“Neutral”  on  the  Fluman  Target  Recognition  (FITR) 
Viewer  and  on  a  Group  Tasking  feature  that  was  expected 
to  help  move  robotic  ground  vehicles.  The  questionnaires 
also  captured  participants’  comments  which  were  used  to 
add,  drop,  or  refine  features  for  future  experiments. 


>  & 


New  Featuresfor  Experiment  3 


Figure  1 1 .  Ratings  of  new  feature  effectiveness 
(1  =  Very  Ineffective,  5  =  Very  Effective). 


Results  on  training  were  based  on  participant 
feedback  on  training  surveys  and  author  observations. 
During  the  course  of  experiments,  the  individual  training 
received  by  the  command  group  participants  improved 
substantially.  Improvements  included  the  development  of 
a  User’s  Manual  to  support  training  and  performance,  and 
the  addition  of  a  dedicated  trainer.  Shortcomings  in  the 
training,  however,  were  observed  and  documented, 
particularly  in  the  area  of  collective  training  across  the 
command  group. 
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Participants  were  more  critical  of  the  training  after 
performing.  For  example,  after  Experiment  4  participants 
expressed  more  concerns  about  training  content  and  time 
after  they  had  completed  the  experimental  runs.  Prior  to 
the  experimental  runs,  and  immediately  after  training, 
their  assessment  of  the  training  was  more  favorable.  This 
finding  seems  an  important  reminder  that  often  “you  don’t 
know  what  you  don’t  know”  until  you  try  to  do  the  job. 

Even  after  Experiment  4,  the  participants  who  had 
completed  40+  runs  across  experiments  expressed  strong 
concerns  about  the  training  requirement  and  their  ability 
to  exploit  Unit  Cell  capabilities.  Overall,  the  training 
results  indicated  that  high  levels  of  tactical  and  technical 
expertise  will  be  required  by  command  groups  equipped 
with  the  automated  and  interdependent  systems  expected 
with  FCS. 

3.4  Automated  Measures 

Automated  measures  of  human-computer  interaction 
are  required  to  provide  efficient  and  effective  measures  of 
command  and  control  performance  for  training  and 
evaluation  (Department  of  the  Army,  2003).  The  manual 
analysis  of  participants’  human-computer  interactions  for 
FCS  C^  underscored  this  requirement.  Identifying  and 
tabulating  the  1,500+  interactions  that  typically  occurred 
during  a  run’s  planning  and  execution  phases  required 
about  16  analyst  workdays. 

A  set  of  23  automated  measures  were  identified  by 
ARJ  to  capture  command  group  interactions  with  the  C^ 
prototypes.  A  goal  was  to  validate  the  automated  data 
obtained  against  ARl’s  manually  derived  data  on 
corresponding  FlCl  measures.  Flowever,  only  3  of  the  23 
automated  measures  requested  by  ARl  were  actually 
developed,  and  only  2  of  those  3  measures  were 
successfully  validated.  This  result  highlights  the  need  for 
“command  emphasis”  to  meet  the  FCS  requirement  for 
automated  performance  assessment. 

3.5  Novice  Versus  Expert  Performance 

A  comparison  of  novice  versus  expert  command 
groups  was  based  on  cadet  performance  during  the 
Summer  Experiment  versus  the  lieutenant  colonel 
performance  during  Experiments  2  and  3.  Applying  the 
same  measurement  approach  to  cadets’  command  group 
performance  resulted  in  clear  and  important  differences 
between  the  two  groups  (Carnahan,  Lickteig,  Sanders,  & 
Durlach,  2004).  Findings  that  seem  likely  to  reflect  true 
novice-expert  differences  included: 

■  Novices  spent  more  time  in  silence,  less  time 

collaborating. 

■  Novices  talked  more  about  firing,  less  about  seeing. 


■  Novices  talked  more  about  own  troops,  less  about 
enemy. 

■  Novices  talked  more  about  enemy  location,  less 
about  enemy  size,  type,  and  disposition. 

■  Novices  performed  fewer  computer  interactions  to 
recognize  and  identify  targets. 

■  Novices  performed  more  computer  interactions  to 
assess  battle  damage. 

These  results  are  consistent  with  other  findings  in  the 
tactical  domain  in  which  novices  demonstrate  greater 
concern  with  friendly  versus  enemy  activities  and  view 
the  enemy  in  a  more  one-dimensional  fashion  (Strater, 
Endsley,  Pleban  &  Matthews,  2000).  The  novice  group 
seemed  to  approach  their  command  and  control  roles  with 
a  hasty  “find  and  kill”  mindset,  not  unlike  a  video  game. 
In  contrast,  the  relatively  expert  command  group 
deliberately  strived  to  build  an  accurate  and  complete 
account  and  understanding  of  the  battlefield  situation. 

4.  CONCLUSIONS 

Conclusions  underscore  potential  problems  in 
training  and  workload  with  FCS,  and  potential  solutions 
through  user-based  involvement  and  proactive  research  to 
ensure  technology  complements  human  performance. 

4.1  Workload 

Human  performance  findings  from  the  FCS  C^ 
Experiments  1-4  indicated  moderate  to  high  levels  of 
workload  for  the  participants.  Overall,  the  findings 
suggest  that  workload  may  be  a  serious  concern  for  future 
command  groups  in  small  FCS  units  in  more  complex 
situations.  Objective  data  confirmed  that  participants 
were  heavily  engaged  in  verbal  and  human-computer 
interaction  during  the  execution  of  more  complex  runs 
and  more  intense  mn  segments.  Subjective  data 
emphasized  the  potentially  negative  impact  of  workload 
based  on  participants’  low  to  moderate  ratings  of  their 
performance  success  in  more  complex  runs. 

Automation  can  reduce  workload,  but  also  increase 
it.  For  Experiment  3,  an  automated  audit  trail  on  sensor 
images  was  introduced  that  indicated  if,  when,  and  by 
whom  each  image  had  been  opened  and  viewed.  The 
audit  trail  helped  reduce  the  number  of  same  images 
reviewed  in  Experiment  2  versus  Experiment  3  from  62% 
to  16%.  A  conclusion  after  Experiment  three,  was  that 
the  decreased  workload  observed,  based  on  objective  HCI 
data  and  subjective  responses,  was  due  to  the  array  of  new 
and  increasingly  automated  features  added  to  the 
prototype  for  Experiment  3. 

However,  increases  in  HCI  frequency  (Figure  7)  and 
perceived  workload  (Figure  9)  occurred  in  Experiment  4 
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despite  more  automated  features  and  a  more  experieneed 
eommand  group.  Perhaps,  programmatic  expeetations 
about  better  baseline  performanee  levels,  ineluding 
requests  to  the  eommand  group  for  faster  run/mission 
eompletion,  raised  performanee  requirements.  Similar 
inereases  in  workload  based  on  expeetations  about  doing 
“more  with  less”  through  teehnology  are  eommon  in  the 
military  (Cordesman  &  Wagner,  1996). 

4.2  Training 

PCS  eommand  groups  will  require  exeeptional  levels 
of  taetieal  and  teehnieal  expertise.  Command  groups  with 
robotie  elements  must  reformulate  battle  eommands  into 
eomputer  eommands.  Traditionally,  sueeinet  military 
eommunieations  like  eommander’s  intent  and  guidanee 
entail  many  implied  tasks  for  manned  systems.  In  the 
future,  eommunieations  with  unmanned  systems  will 
require  more  explieit  and  preeise  speeifieation  in 
eomputer-mediated  and  -dietated  formats. 

Teehnieal,  and  even  taetieal,  expertise  may  also 
quiekly  degrade  as  and  PCS  systems  ehange  rapidly  in 
response  to  ehanges  in  teehnology  and  operational 
requirements. 

Understanding  the  limits  and  strengths  of  notional 
PCS  systems  and  the  C^  system  that  eontrolled  them  was 
a  severe  ehallenge.  The  partieipants  in  the  eommand 
group  eontributed  heavily  to  the  design  of  the  C^ 
prototype,  eondueted  numerous  runs  aeross  experiments, 
and  had  higher  rank  and  more  operational  experienee  than 
expeeted  for  future  PCS  small-unit  leaders.  Despite  these 
advantages,  the  eommand  group  eontinued  to  report 
diffreulty  in  understanding  the  input  requirements  and 
operational  eonsequenees  for  highly  automated  and 
interrelated  Plan,  See,  Move,  and  Strike  funetions  aeross 
the  four  experiments. 

The  partieipants’  parting  responses  on  training,  after 
their  fourth  experiment  and  40+  runs,  underseored  the 
training  ehallenge  for  future  eommand  groups  in  units 
with  the  automated  and  interdependent  systems 
antieipated  for  PCS: 

■  Need  more  hands-on  in  taetieal  seenarios  or  vignettes, 
less  leeture.  Putting  the  lesson  in  a  taetieal  situation 
really  lends  eredenee  to  the  funetion  you  are  teaehing 
and  demonstrates  why  you  want  to  learn  it  and  how  it  is 
best  employed. 

■  Training  should  be  more  dedieated  to  aetual 
employment  teehniques.  We  eould  have  used  more 
time  integrating  as  a  team. 

■  We  get  more  funetional  every  run,  what  we  need  are  a 
eouple  of  runs  designed  for  us,  not  to  be  eritiqued  (i.e., 
in  AAR),  but  for  us  to  re-establish  SOPs. . ..  Not  an  easy 
task.... 


Training  is  the  glue  that  will  hold  PCS  and  the  Puture 
Poree  together.  Conelusions  by  ARJ  stress  the  need  for 
three  basie  but  far-reaehing  improvements  in  the  training 
for  PCS,  partieularly  eommand  groups: 

■  Develop  progressive  and  simulation-based  training 
exereises  direeted  at  individual  teehnieal  and  taetieal 
skills,  espeeially  user  input  requirements  for,  and  the 
eonsequenees  of,  automated  funetions. 

■  Develop  a  parallel  set  of  eolleetive  training  exereises 
for  ;ntra-unit  teehnieal  and  taetieal  skills. 

■  Develop  a  parallel  set  of  multi-eehelon,  distributed 
training  exereises  for  cro^^-unit  teehnieal  and  taetieal 
skills. 

4.3  Warfighter  Involvement 

The  C^  interfaee  is  inereasingly  the  primary  loeus,  or 
means,  of  interaetion  between  Soldiers  and  systems. 
Developing  an  optimal  interfaee  requires  sustained  and 
intense  warfighter  involvement. 

A  distinetive  hallmark  of  the  PCS  C^  program  is  the 
eommitment  by  the  same  partieipants  over  two  years  and 
four  experiments  to  use  and  refine  a  prototype  C^  system. 
The  program  eontinues  to  develop  and  refine  a  eutting- 
edge  interfaee  to  eommand  and  eontrol  an  PCS  equipped 
foree.  Key  aspeets  of  the  program’s  prototype  C^  system 
and  interfaee  inelude: 

■  Value  based  on  military  experts’  sustained  use. 

■  Viable  interfaee  to  networked/unmanned  systems. 

■  Common  interfaee  to  manned/unmanned  systems. 

4.4  Proactive  Research 

Plistorically,  Army  acquisition  research  has  had 
difficulty  conducting  an  adequate  early  assessment  of  the 
human  dimension  in  system  performance.  This  human 
performance  issue  is  especially  critical  for  PCS  because 
the  empowerment  of  the  commander  through  advanced  C^ 
systems  is  at  the  heart  of  the  PCS  concept. 

Moreover,  the  revolutionary  nature  of  the  Army’s 
transformation  embodied  in  the  PCS  acquisition  program 
increases  the  risk  of  relying  exclusively  on  traditional 
assessment  methods  such  as  C^  hardware  and  software 
component  tests,  or  the  outcomes  of  simulation  without 
Soldi  ers-in-the-loop. 

The  PCS  C^  program,  now  expanded  to  MC&D  C^, 
exemplifies  the  proactive  research  on  human  performance 
that  is  essential  to  forging  the  human-machine  alliance 
required  by  PCS.  ARI’s  methods  and  results  on  human 
performance  provide  reliable  and  empirical  data  for 
important  and  timely  decisions  on  training,  materiel, 
manpower,  and  personnel. 


7 


The  human  performance  findings  summarized  here 
were  readily  transitioned  to  PCS  acquisition  efforts,  in 
part  due  to  DARPA’s  dual  roles  in  PCS  simulation  and 
acquisition.  The  PCS  Integrated  Product  Team  for 
Training  is  using  the  human  performance  findings  in  their 
plans  and  designs.  Peedback  based  on  the  findings  helped 
shape  the  prototype  showcased  in  the  Capstone 
Demonstration  of  C^  systems  prior  to  PCS  Milestone  B. 
The  analysis  of  command  group  tasks  and  human- 
computer  interactions  by  ARI  was  provided  to  the  Lead 
System  Integration  Package  27  Vendors  for  PCS. 

In  closing,  two  additional  conclusions  are  stressed. 
The  ultimate  value  of  a  C^  research  and  development 
program  is  determined  as  much  by  the  investment  in 
training  and  evaluation,  as  investment  in  simulation.  The 
ultimate  value  of  a  C^  system  is  determined  not  so  much 
by  technology,  but  by  shaping  technology  to  complement 
human  performance. 


REFERENCES 

Carnahan,  T.J.,  Lickteig,  C.W.,  Sanders,  W.R.,  &  Durlach,  P.  J.  (2004). 
Novice  versus  expert  command  groups:  Preliminary  findings  and 
training  implications  for  future  combat  systems  (ARI  Research 
Report  1821).  Alexandria,  VA:  U.S.  Army  Research  Institute  for 
the  Behavioral  and  Social  Sciences. 

Cordesman,  A.  H.  and  Wagner,  A.  R.,  (1996).  The  lessons  of  modern 
war,  Volume  IV:  The  Gulf  war.  Boulder,  CO:  Westview  Press. 

Department  of  the  Army.  (2003,  April  14).  Operational  requirements 
document  for  the  Future  Combat  Systems  (Change  3,  JROC 
Approved).  Fort  Knox,  KY :  Unit  of  Action  Maneuver  Battle  Lab. 

Lickteig,  C.  W.,  Sanders,  W.  R.,  Durlach,  P.  J.,  Lussier,  J.  W.  & 
Carnahan,  T.  J.  (2004a).  Human  Performance  Essential  to  Battle 
Command:  Report  on  Four  Future  Combat  Systems  Command 
and  Control  (FCS  (f)  Experiments  (ARI  Research  Report  1812). 
Alexandria,  VA:  U.  S.  Army  Research  Institute  for  the  Behavioral 
and  Social  Sciences. 

Lickteig,  C.  W.,  Sanders,  W.  R.,  Durlach,  P.  J.,  &  Carnahan,  T.  J. 
(2004).  Future  combat  systems  command  and  control  (FCS  C^) 
human  functions  assessment:  Experiment  3  interim  report  (ARI 
Research  Report  1819).  Alexandria,  VA:  U.S.  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences. 

Strater,  L.  D.,  Endsley,  M.  R.,  Pleban,  R.  J.,  &  Matthews,  M.  D.  (2000). 
Measures  of  platoon  leader  situation  awareness  in  virtual 
decision-making  exercises  (ARI  Research  Report  1770). 
Alexandria,  VA:  U.S.  Army  Research  Institute  for  the  Behavioral 
and  Social  Sciences. 


8 


